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Abstract  We  (jeflne  and  discuss  a  class  of  multi-robot  systems  possessing  ergodic  dy¬ 

namics  and  show  that  they  are  realizable  on  physical  hardware  and  useful  for  a 
variety  of  tasks  while  being  amenable  to  analysis.  We  describe  robot  controllers 
synthesized  to  possess  these  dynamics  and  also  physics-based  methodologies 
that  allow  macroscopic  structures  to  be  uncovered  and  exploited  for  task  execu¬ 
tion  in  systems  with  large  numbers  of  robots. 
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1.  Introduction 

Multi-robot  systems  can  both  enhance  and  expand  the  capabilities  of  single 
robots,  but  robots  must  act  in  a  coordinated  manner.  So  far,  examples  of  co¬ 
ordinated  robot  systems  have  comprised  of  largely  domain-specific  solutions, 
with  few  notable  exceptions.  In  this  paper  we  describe  our  ongoing  work  on 
the  development  of  formal  methodologies  for  synthesis  of  multi-robot  systems 
that  address  these  issues  in  a  principled  fashion. 

We  focus  here  on  inter-robot  dynamics,  the  roles  played  by  those  dynam¬ 
ics  toward  task  achievement,  and  their  implications  in  feasible  formal  methods 
for  synthesis  and  analysis.  We  describe  automated  synthesis  of  controllers 
that  capitalize  on  so-called  ergodic  dynamics,  which  enable  mathematical  ar¬ 
guments  about  system  behavior  to  be  simplified  considerably.  Sensor-based 
simulations  and  physical  robot  implementations  show  that  these  controllers  to 
be  feasible  for  real  systems.  We  further  suggest  that  this  approach  will  scale  to 
systems  with  large  numbers  of  robots. 

2.  Related  formal  methodologies 

Formal  methodologies  for  synthesis  and  analysis  of  multi-robot  systems  dif¬ 
fer  based  on  the  type  of  systems  they  aim  to  address.  One  successful  method 
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Figure  1  The  Markovian  world  as  states  with  action  tran¬ 
sitions.  The  sequential  task  specification  is  a  particular  se¬ 
quence  of  world  transitions.  The  task  is  shown  here  shaded 
within  the  world  specification. 


for  analysis  of  swarm  systems  is  based  on  the  theory  of  stochastic  processes: 
for  example,  in  the  phenomenological  modeling  and  analysis  of  multi-robot 
cooperative  stick-pulling,  a  macroscopic  difference  equation  for  the  rates  of 
change  of  each  type  of  robot  state  is  derived  from  the  stochastic  master  equa¬ 
tion  and  sensor-based  simulations  are  used  to  estimate  parameter  values  (Mar- 
tinoli  et  al.,  2004).  An  extension  to  the  same  theory  (but  using  continuous 
differential  equations  instead)  allows  adaptive  systems  to  be  modeled  (Lerman 
and  Galstyan,  2003).  When  applied  to  foraging,  the  analysis  enabled  system 
design  improvements.  Explicitly  coordinated  systems  arc  typically  addressed 
at  the  algorithmic  level,  such  as  in  the  Computation  and  Control  Language 
(Klavins,  2003)  and  formal  studies  of  multi-robot  task  allocation  methodolo¬ 
gies  (Gerkey  and  Mataric,  2004).  Also  related  is  Donald’s  (1995)  Information 
Invariants  Theory,  and  Erdmann’s  (1989)  studies  of  the  advantages  of  proba¬ 
bilistic  controllers. 


3.  Behavioral  configuration  space  and  ergodicity 

The  physical  configuration  space,  common  in  robotics  for  representing  phys¬ 
ical  arrangements,  can  be  augmented  to  include  additional  dimensions  for  each 
of  the  robot’s  internal  control  variables  that  observable  behavior.  We  call  this 
the  behavioral  configuration  space  (BCS).  It  is  a  useful  mental  representation 
for  a  multi-robot  system  and  for  reasoning  about  the  overall  system  dynamics. 
For  practical  applications,  we  will  only  consider  particular'  subspaces,  never 
the  full  configuration  space. 

The  BCS  of  a  single  robot  consists  of  dimensions  for  the  physical  config¬ 
uration  (e.g,.  the  pose  variables,  and  velocities  if  necessary)  and  dimensions 
for  internal  state  variables  (continuous  or  discrete  values  within  memory).  The 
range  of  each  dimension  is  determined  by  constraints  on  state  variables.  The 
BCS  of  an  ensemble  of  robots  is  constructed  from  essentially  a  Cartesian  prod¬ 
uct  of  individuals  spaces  and  the  spaces  of  movable  obstacles,  etc.  The  con¬ 
straints  (e.g.,  two  robots  simultaneously  occupying  the  same  location)  subtract 
components  from  this  product.  Couplings  between  the  robots  via  communica¬ 
tion  channels,  mutual  observation,  etc.,  further  restrict  this  space. 

The  global  state  of  the  multi-robot  system  at  any  specific  time  can  be  rep¬ 
resented  by  a  point  in  BCS  and  likewise,  the  time-evolution  of  the  system, 
as  a  trajectory.  A  system  that  exhibits  ergodic  dynamics  completely  visits  all 
parts  of  the  configuration  space  with  probability  that  is  dependent  only  on  the 
volume  of  that  part  of  the  space.  Long  term  history  is  unimportant  in  predict- 
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ing  the  dynamical  behavior  because  the  system  “forgets”  previous  trajectories. 
Time  averages  of  some  system  property  (over  a  duration  longer  than  the  under¬ 
lying  dynamics  timescale),  arc  equal  to  (configuration)  spatial  averages.  Few 
useful  robotics  systems  arc  entirely  ergodic,  but  various  sub-parts  of  the  BCS 
may  be  ergodic.  The  next  section  describes  one  such  system. 

4.  Automated  synthesis  for  sequential  tasks 

Jones  and  Mataric  (2004a,  2004b)  have  developed  a  framework  for  auto¬ 
matic  and  systematic  synthesis  of  minimalist  multi-robot  controllers  for  se¬ 
quential  tasks.  The  framework  consists  of  a  suite  of  algorithms  that  take  as  in¬ 
put  a  formal  specification  of  the  environmental  effects,  the  task  requirements, 
and  the  capabilities  of  the  robots.  The  algorithms  produce  either  provably  cor¬ 
rect  robot  controllers,  or  point  to  the  exact  scenarios  and  task  segments  which 
make  (algorithmically)  guaranteed  task  completion  impossible.  The  type  of 
controller  and  prospect  of  successful  task  execution  depend  on  the  capabili¬ 
ties  of  the  individual  robots.  Current  options  include  the  possibly  of  broadcast 
inter-robot  communications  (Jones  and  Mataric,  2004a),  and  a  local  memory 
on  each  of  the  robots  permitting  non-reactive  controllers  (Jones  and  Mataric, 
2004b).  Two  complementary  analysis  techniques  allow  various  statistical  per¬ 
formance  claims  to  be  made  without  the  cost  of  a  full  implementation  and 
exhaustive  experimentation. 

We  do  not  provide  full  details  of  the  framework  here,  but  instead  focus  on 
the  (non-obvious)  role  of  ergodic  dynamics  in  the  work.  The  framework  uses  a 
set  of  states  S  to  denote  the  possible  states  that  the  (assumed  to  be  Markovian) 
world  can  be  in.  The  set  A  contains  actions  which  act  upon  the  world  state, 
producing  state  transitions  defined  in  some  probabilistic  manner  (see  Figure  1). 
A  particular-  sequence  of  states,  say  T,  (T  C  S )  makes  up  the  task.  In  actuality 
the  robots  are  only  interested  in  producing  the  single  task  sequence,  and  thus 
only  those  transitions  need  to  be  stored.  Thus,  S  is  never  stored  or  calculated, 
only  T  need  be  kept,  and  |Tj  <C  .S j .  Robots  then  move  around  the  environment 
making  observations,  perhaps  consulting  internal  memory  or  listening  to  the 
broadcast  communication  channel  if  suitably  equipped.  If  a  robot  has  sufficient 
information  to  ensure  that  the  performance  of  a  particular-  action  (from  A)  can 
only  result  a  world  transition  that  is  part  of  the  task  (i.e.,  result  in  a  state  in  T) 
then  it  may  perform  that  action. 

Returning  to  the  notion  of  behavioral  configuration  space,  each  of  the  world 
states  in  S  represents  entire  subspaces  of  the  overall  system's  space.  Figure  2 
shows  that  the  entire  behavioral  configuration  space  as  it  fits  into  this  formal¬ 
ism.  A  hypothetical  projection  of  this  entire  (huge)  configuration  space  sep¬ 
arates  the  configurations  into  subspaces,  each  subspace  representing  a  single 
state  in  T.  Actions  (from  A)  evolve  the  world  state  and  hence  transition  the 
system  from  one  subspace  to  another.  We  design  the  system  so  that  within  each 


Figure  2  Three  different  views  of  the  system  dynamics: 
(a)  A  trajectory  within  the  entire  behavioral  configuration 
space;  (b)  A  hypothetical  projection  of  the  configuration 
space  showing  planar  subspaces  for  behavior  not  part  of  S , 
and  actions  from  A  connecting  these  spaces;  (c)  The  ab¬ 
stracted  S  representation  as  used  by  the  described  formal¬ 
ism. 


subspace  the  dynamics  are  ergodic.  Work  that  has  used  this  formal  framework 
ensured  this  property  by  having  the  robots  perform  randomized  exploration 
policies.  The  randomized  strategy  needs  to  have  sufficient  effect  to  overpower 
other  systematic  biases  in  the  system  that  could  produce  large  scale  effects  and 
ignore  some  part  of  the  configuration  space. 

Both  controllers  with  memory  (Jones  and  Mataric,  2004b)  and  ones  en¬ 
dowed  with  communication  capabilities  (Jones  and  Mataric,  2004a)  were  demon¬ 
strated  in  simulation  and  on  physical  hardware  in  a  multi-robot  construction 
domain.  The  task  involves  the  sequential  placement  of  colored  cubes  into  a 
planar  arrangement.  The  sequence  T  contains  simply  the  required  evolution  of 
the  structure,  actions  A  being  the  placement  of  an  individual  cube.  Referring 
back  to  Figure  2;  in  the  construction  domain  the  motions  within  each  subspace 
arc  random  walks  by  the  robots,  and  the  transitions  between  spaces  arc  cube 
placement  actions. 

Analytical  techniques  developed  in  order  to  predict  task  execution  arc  aided 
by  the  ergodic  components  of  the  robots  behavior  in  this  domain.  One  example 
is  in  the  macroscopic  model  (Jones  and  Mataric,  2004b,  pp.  4-5)  applied  to 
this  formal  framework.  This  model  calculates  the  probability  of  successful  task 
completion  by  calculating  a  large  multiplication  of  all  possible  memory  states 
that  get  set,  in  each  possible  world  state,  after  each  possible  observation,  calcu¬ 
lating  the  probability  that  only  the  correct  action  will  result  and  includes  terms 
for  when  actions  may  result  in  other,  or  null,  world  transitions.  A  fundamental 
assumption  for  that  calculation  is  that  no  “structure”  in  the  world  results  in  the 
observation  and  action  sequences  that  correlate.  When  endowed  with  naviga¬ 
tional  controllers  that  have  ergodic  dynamics,  we  know  that  this  is  true  because 
the  observation  of  an  ergodic  system  at  N  arbitrary  instants  in  time  is  statisti¬ 
cally  the  same  as  N  arbitrary  points  within  the  behavioral  space  (McQuarrie, 
1976,  pp.  554). 

This  section  has  demonstrated  that  dynamics  with  a  high  degree  of  ergodic- 
ity  arc  achievable  on  physical  robot  systems.  They  can  play  a  role  in  systems 
for  which  analytical  methods  exist,  and  as  a  very  simple  form  of  dynamics  they 
can  aid  in  simplifying  particular  aspects  of  system  design. 
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5.  Large-scale  multi-robot  systems 

We  consider  large-scale  multi-robot  systems  those  with  robots  on  the  order 
of  thousands.  In  spite  of  the  fact  that  manufacturing  and  tractable  simulation 
remain  open  challenges,  a  variety  of  tasks  have  been  proposed  for  systems 
of  this  type.  Increasing  the  number  of  robots  increases  the  total  number  of 
degrees-of-freedom  in  a  system,  and  results  in  a  highly  dimensional  BCS.  Co¬ 
ordination  approaches  that  couple  robot  interactions  as  loosely  as  possible  arc 
most  likely  to  scale  to  large  sizes. 

Mathematical  techniques  employed  in  statistical  mechanics  arc  useful  for 
establishing  the  relationship  between  microscopic  behavior  and  macroscopic 
structures  (McQuarrie,  1976).  Typical  system  sizes  for  classical  work  arc  sig¬ 
nificantly  larger  (~  1023)  than  the  numbers  currently  conceivable  for  robots.  In 
the  case  of  large  (or  infinite)  systems,  interesting  macroscopic  structures  can 
result  even  from  ergodic  local  dynamics,  global  structures  like  equilibrium 
phases,  phase  transitions,  coexistence  lines,  and  critical  points  are  widely  stud¬ 
ied  in  thermodynamics.  Recent  work  attempts  to  reformulate  many  of  these 
classical  notions  for  systems  with  fewer  entities  (Gross,  2001). 

We  arc  pursuing  a  methodology  for  coordination  of  large-scale  systems 
through  the  study  of  a  small  set  of  mechanisms  for  producing  general  macro¬ 
scopic  phenomena.  One  candidate  mechanism  is  a  protocol  for  achieving  con¬ 
sensus.  The  Potts  (1952)  model  is  illustrative;  it  is  an  archetypal  magnetic  spin 
system  that  models  interactions  between  particles  at  a  number  of  fixed  loca¬ 
tions  within  a  graph  or  lattice.  The  Ising  model  (a  specific  Potts  model)  has 
also  been  used  to  model  gas  flow.  Neither  model  is  a  perfect  fit  for  robots,  but 
illustrates  macroscopic  structure  from  simple  interactions. 

Mapping  the  spin  interactions  at  spin  sites  to  robots  allows  for  the  develop¬ 
ment  of  a  communication  algorithm  that  possesses  ergodic  dynamics  (and  an 
energy  conservation  constraint)  that  permits  the  definition  of  a  partition  func¬ 
tion  Z  that  can  be  solved  using  a  numerical  method  for  pseudo-dynamics  sim¬ 
ulation  (or  in  trivial  cases  analytically).  This  admits  a  prediction  of  global  be¬ 
havior  because  exhaustive  parameter  variations  enable  construction  of  a  phase 
diagram.  In  the  case  of  the  Potts  and  Ising  models  this  phase  diagram  is  well 
known.  Particular  regions  of  the  phase  space  in  the  Ising  model  represent  re¬ 
gions  of  maximal  order.  For  robots  this  means  unanimity;  consensus  is  reached 
through  a  second-order  phase  transition. 

The  ability  to  prescribe  ergodic  dynamics  for  large-scale  robot  systems  makes 
those  analytical  approaches  that  focus  only  on  constraint  space  topology  fea¬ 
sible  for  predictions  of  global  structure.  This  means  that  task  directed  actions 
can  be  tackled  directly  from  a  macroscopic  perspective. 
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6.  Summary  and  Conclusion 

We  have  taken  a  dynamics-centric  approach  to  describing  multi-robot  be¬ 
havior.  This  view  has  suggested  that  the  notion  of  ergodicity  may  be  useful 
within  a  robotics  context,  something  that  we  have  demonstrated  throughout 
the  paper.  After  defining  a  behavioral  configuration  space,  we  demonstrated 
that  subspaces  in  which  the  robot  dynamics  arc  essentially  ergodic  can  be  used 
to  produce  meaningful  behavior,  and  allow  automated  synthesis  techniques  to 
focus  on  a  small  set  of  task-oriented  states,  rather  then  the  entire  ensemble 
configuration  space.  Also,  in  at  least  one  case,  ergodicity  simplifies  analysis  of 
system  behavior.  Implementations  on  physical  and  simulated  robots  show  that 
ergodicity  is  indeed  achievable  in  the  real  world.  Future  promise  of  this  general 
approach  is  suggested  in  a  discussion  of  large-scale  multi-robot  systems. 
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